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STUDY  ON  EXTREMIZING  ADAPTIVE  SYSTEMS  AND  APPLICATIONS  TO 
SYNTHETIC  APERTURE  RADARS 

Brief  Interim  Report 


ABSTRACT 

^’Klopf's  work  ^on  the  functioning  of  the  neuron  was 
studied  and  critically  examined  for  engineering  application 
possibilities.  Similarly,  Barto's  work  *>n  the 
implementation  of  Klopf's  ideas  in  computer  simulated 
nets/systems  was  studied  to  determine  if  it  could  provide 
suitable  models  for  physical  systems.  The  latest  learning 
system  Investigated  by  Barto,  described  as  ‘Learning  with  an 
Adaptive  Critic*  -tyf  was  considered  as  the  most  promising  for 
engineering  applications.  A  functional  engineering  model  of 
that  system  has  been  developed  and  its  dynamic  behavior  of 
this  system  Is  currently  being  investigated  in  order  to 
Improve  our  understanding  of  the  system  operation  and 
potential  applications.  In  parallel  with  this  study  we  are 
looking  for  possible  application  of  such  learning  systems  in 
synthetic  aperture  radars  'fSA*T~and  data  exploitation. 
Several  potential  applications  have  already  been  suggested. 
These  suggestions  will  be  further  explored  and  the  most 
promising  will  be  proposed  for  full  Investigation  and  possible 
Implementation.  In  FY84. 

A 

This  project  was  motivated  by  A.  H.  Klopf's  Insightful  observation 
and  proposition  on  the  functioning  of  the  neuron  cell  and  the  nervous 
system  In  general,  and  the  work  done  by  Professor  A.  Barto  and  his 
associates  at  the  University  of  Massachusetts  In  an  effort  to  design  and 
computer  simulate  networks,  and  systems  of  networks,  operating  on  the 
principles  proposed  by  Klopf.  We  believe,  and  hope  to  show  eventually 
that  physical  systems  can  be  built,  and  applications  In  physical  systems 
can  be  found  for  such  systems  that  will  yield  superior  performance. 

Task  1  of  the  project  required  that  first  we  study  carefully 

Klopf's  work  clearly  presented  In  Ref  [1]  In  order  to  fully  understand 

and  appreciate  Its  Implications  and  then  do  the  same  with  Barto's  work 
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[2].  The  purpose  there  is  to  understand  the  various  implementation 
models  developed  and  select  the  most  promising  one  for  application  to 
synthetic  aperture  radars. 

We  believe  that  Klopf's  fascinating  work  described  in  Ref.  1  opens 
up  new  ways  of  thinking  for  designing  physical  systems  which  challenge 
our  imagination,  but  may  take  a  little  time  until  we  learn  how  and  where 
to  best  implement  them. 

Barto's  work  [2]  Is  an  excellent  study  on  adaptation  and  learning 
problems  and  learning  rules,  with  special  emphasis  given  to  Klopf's 
heterostat.  In  a  system  operating  in  accordance  with  this  reinforcement 
learning  rule,  the  weighting  function  at  the  i-th  input,  W^t),  is 
enhanced  if  excitation  of  the  1-th  input  at  t  leads  to  excitation  at  the 
1-th  input  r' seconds  later,  the  enhancing  function  efc)  decaying 
exponentially  with  r.  Thus  «  is  a  cause-effect  measure,  a  small  « 
Indicating  a  strong  "link"  between  the  i-th  input  and  the  output  it 
produces. 

Several  goal -seeking  systems  of  goal -seeking  components  were 
developed  and  studied  by  Barto.  Of  these  systems,  those  described  as 
“learning  with  a  critic"  are  judged  to  be  potentially  more  applicable  to 
our  engineering  world. 

Systems  described  as  "learning  with  a  teacher"  require  that  the 
controller  “knows  the  answers  to  a  set  of  questions",  i.e.,  knows  what 
the  response  to  a  set  of  Inputs  should  be  and  provides  the  system  with 
appropriate  corrective  signals.  In  most  engineering  systems,  this 
operation  requires  more  information  than  Is  usually  available.  Learning 
with  a  critic,  on  the  other  hand,  requires  onljkthat  an  observation  be 
made  as  to  whether  the  output  is  changing  in  the  right  direction.  One 
could  reasonably  expect  that  such  Information  should  be  available  in 
many  engineering  applications. 
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It  seems  to  us  that  learning  with  a  critic  encompasses  the  class  of 
problems  where  the  desired  goal  is  to  reduce  a  measured  error  signal 
without  identifying  contributing  factors  and  system  dynamics.  This 
covers  indeed  a  large  class  of  problems. 

Investigation  of  Dynamics  of  Adaptive  Nets 

The  most  promising  system  for  engineering  applications  developed  by 
Barto  was  judged  to  be  the  one  containing  an  adaptive  critic  in  the 
feedback  path  and  reported  In  Ref.  [3]. 

The  controlled  dynamic  system,  again,  is  the  cart-pole  system, 
except  that  the  controller  action  is  adaptive  with  its  weighting 
function  adjusted  by  the  Adaptive  Critic  Element  and  system  output,  plus 
an  eligibility  function  mechanism  resident  in  the  controller.  Because 
this  system  has  shown  a  much  improved  learning  capability,  we  chose  it 
as  the  model  to  Investigate  its  dynamic  behavior.  Later  the  system  may 
be  modified  to  bring  it  into  closer  agreement  with  engineering 
applications  in  general  and  synthetic  aperture  radar  system  requirements 
In  particular. 

It  was  decided  to  carry  out  the  analysis  of  the  above  system  in  the 
following  sequence  of  steps: 

a)  First  analyze  the  system  with  the  Adaptive  Search 
Element  only,1.e.,  without  the  Adaptive  Critic  Element 
in  the  loop.  Develop  a  block  diagram  which  shows  the 
feedback  structure  of  the  ASE  and  any  nonlinearity, 
nonstationarlty  and  sampling; 

b)  study  the  control  nature  of  the  ASE  and  alternate  forms 
of  the  ASE  feedback; 


c)  define  what  is  meant  by  stability  for  this  type  of 
system; 


d)  reduce  the  ASE  to  a  linear  stationary  control  problem 
and  analyze  stability; 

e)  add  sampling  to  the  system  discussed  previously  and 
Investigate  the  Inpact  on  stability; 

f)  add  nonlinearities  to  the  system  and  Investigate  the 
Impact  on  stability;  draw  conclusions; 

g)  verify  conclusions  by  using  a  computer  simulation  of 
ASE,  and 

h)  add  the  ACE  and  Investigate  Its  effect  on  stability. 

The  above  work  Is  proceeding  and  we  hope  It  will  be  concluded  by 
the  end  of  the  first  year  effort. 

Task  3  calls  for  identifying  potential  SAR  applications  of  adaptive 
goal -seeking  systems. 

In  order  to  generate  as  many  ideas  as  possible,  a  presentation  of 
the  Adaptive  System  was  made  to  a  select  group  of  the  Radar  Division 
technical  staff.  As  a  result  of  this  meeting,  several  ideas  were 
proposed  for  possible  application.  These  Ideas  will  be  further  explored 
and  the  most  promising  will  be  proposed  for  investigation  and 
implementation  in  the  next  year. 

References 

1.  A.  H.  Klopf,  The  Hedonistic  Neuron,  Hemisphere  Publishing  Corp., 
Washington,  1962. 

2.  Andrew  Barto,  "Goal  Seeking  Components  for  Adaptive  Intelligence: 

An  Initial  Assessment",  Technical  Report  AFWAL-TR-81-1070,  CIS 
Dept.,  University  of  Massachusetts,  April  1981. 

3.  A.  Barto,  R.  Sutton,  C.  Anderson,  "Neuron-Like  Adaptive  Elements 
that  Can  Solve  Difficult  Learning  Control  Problems",  CIS  Dept., 
University  of  Massachusetts,  Report  No.  82-20. 


